Non-rigid Self-calibration of a Projective Camera

Rigid structure-from-motion (SfM) usually consists of two steps: First, a projective reconstruction is computed which is then upgraded to Euclidean structure and motion in a subsequent step. Reliable algorithms exist for both problems. In the case of non-rigid SfM, on the other hand, especially the Euclidean upgrading has turned out to be difficult. A few algorithms have been proposed for upgrading an affine reconstruction, and are able to obtain successful 3D-reconstructions. For upgrading a non-rigid projective reconstruction, however, either simple sequences are used, or no 3D-reconstructions are shown at all. In this article, an algorithm is proposed for estimating the self-calibration of a projectively reconstructed non-rigid scene. In contrast to other algorithms, neither prior knowledge of the non-rigid deformations is required, nor a subsequent step to align different motion bases. An evaluation with synthetic data reveals that the proposed algorithm is robust to noise and it is able to accurately estimate the 3D-reconstructions and the intrinsic calibration. Finally, reconstructions of a challenging real image with strong non-rigid deformation are presented.